Skip to content

fix(artifacts): keep eval runs at timestamp root - #1580

Merged
christso merged 2 commits into
mainfrom
target-layout-implementation
Jun 30, 2026
Merged

fix(artifacts): keep eval runs at timestamp root#1580
christso merged 2 commits into
mainfrom
target-layout-implementation

Conversation

@christso

Copy link
Copy Markdown
Collaborator

Summary

Finalize the v1 run artifact layout so a timestamped run bundle owns one root index.jsonl manifest and one root summary.json. New eval runs no longer fan out into target or variant directories; target/model/variant remain metadata, and per-result artifacts stay discoverable through explicit manifest fields like result_dir, summary_path, grading_path, metrics_path, and transcript_path.

What Changed

  • Replaced the CLI target/variant bundle writer with a single run-root manifest writer in agentv eval.
  • Updated resume/final artifact writing to aggregate from the run root and preserve allocated per-result directories.
  • Added ADR 0012 documenting the finalized layout, opaque allocated result directories, legacy nested-manifest reader compatibility, and the future internal/ v2 option.
  • Updated result artifact docs, concepts, and layout tests to lock the root-manifest contract.

Code Review

  • Reviewed branch diff against origin/main with correctness, testing, maintainability, project-standards, agent-native, and learnings lenses.
  • No actionable findings.
  • Residual risk: writer integration coverage is updated but could not be executed in this checkout because missing dependencies prevent the suite from starting.

Validation

  • bun test apps/cli/test/commands/eval/result-layout.test.ts passed.
  • git diff --check passed.
  • Attempted bun test apps/cli/test/commands/eval/artifact-writer.test.ts; it failed before running tests because @agentv/core could not be resolved.
  • Attempted bun test apps/cli/test/eval.integration.test.ts; it failed before running tests because execa could not be resolved.
  • Did not run bun install per repo instructions.

Post-Deploy Monitoring & Validation

No production service monitoring is needed. After merge, validate by running a fresh eval and confirming .agentv/results/<experiment>/<timestamp>/index.jsonl is the only new run manifest, with per-result sidecars resolved through row path fields and no target/model/variant/cases folder layer.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jun 30, 2026

Copy link
Copy Markdown

Deploying agentv with  Cloudflare Pages  Cloudflare Pages

Latest commit: 24b9912
Status: ✅  Deploy successful!
Preview URL: https://24327a53.agentv.pages.dev
Branch Preview URL: https://target-layout-implementation.agentv.pages.dev

View logs

@christso
christso merged commit 13b0f9a into main Jun 30, 2026
8 checks passed
@christso
christso deleted the target-layout-implementation branch June 30, 2026 12:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant